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(54) Remote controlling method and apparatus 

(57) Control of a GUI screen at a server in a remote 
location is obtained while the prior instalment of special 
remote controlling software in a remote controlling ma- 
chine is not required, and remote controlling is per- 
formed by using the obtained software. 

Remote controlling software is in an HTML (Hyper- 
Text Markup Language) file that is held by a server, and 
a remote controlling machine downloads it by using a 
web browser. The downloaded remote controlling soft- 
ware is automatically activated and sends a request for 
the activation of remotely controlled software that is 
present in a remotely controlled server. The remote con- 
trolling machine then activates an event monitor an 
event sender, a drawing command receiver and a draw- 
ing command analyser which are functions of the re- 
mote controlling software. 
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Description 

The present invention relates to a remote control- 
ling method and apparatus, and in particular to a method 
for controlling a GUI (Graphical User Interface) screen 
at a remotely controlled machine on a network. 

Conventionally, when remote controlling of a GUI 
screen at a machine in a remote location is to be exer- 
cised, special remote controlling software must be in- 
stalled in advance both in a machine to be remote con- 
trolled and in a machine to exercise the remote control- 
ling. 

Many of the remote controlling machines that are 
currently available are designed for operation in a vari- 
ety of machine and software environments. Some ma- 
chines run under different Operating Systems (OSs). 
Therefore, a great expenditure of effort is required to de- 
velop remote controlling software and install it on all 
these machines. 

In addition, since the same software versions must 
be installed on remotely controlled machines and on 
those that exercise the remote control, a great expend- 
iture of effort is also required to manage these ma- 
chines. For example, before installing an upgrade ver- 
sion, the versions on all the remote controlling machines 
must be examined. 

If remote controlling software is developed for spe- 
cific platforms, the use of the software is necessarily lim- 
ited to those platforms for which it is developed, and the 
multi -platform operations that users demand can not be 
implemented. 

When remote controlling software is routinely in- 
stalled in remote controlling machines, for its storage, a 
certain amount of memory must be permanently allocat- 
ed, and this constitutes a barrier to the downsizing of 
portable remote controlling machines. 

Japanese Examined Patent Publication No. Hei 
8-1624 and Japanese Examined Patent Publication No. 
Hei 7-34188 are prior art related to the present inven- 
tion. In Japanese Patent Examined Publication No. Hei 
8-1624 a technique is disclosed whereby not only can 
a program run on different OSs. but also GUI sources 
are collectively displayed on a screen of a computer sys- 
tem at a remote location. In Japanese Examined Patent 
Publication No. Hei 7-341 88 a computer system for sup- 
porting multi-graphics sessions is disclosed that com- 
prises a central processor, a graphic terminal and a dis- 
play controller processor at a remote area. These con- 
ventional techniques, however, do not resolve the above 
described problems because they involve the prior in- 
troduction of remote controlling software into remote 
controlling machines. 

It is, therefore, one object of the present invention 
to provide a computer system wherein, even when no 
remote controlling software is currently available at a re- 
mote controlling terminal, remote controlling of a server 
can be effected simply by accessing it. 

It is another object of the present invention to sim- 



plify the upgrading operation, for remote controlling soft- 
ware held by a remote controlling terminal, that accom- 
panies the upgrading of the version of the remote con- 
trolling software held by a server that is to be remotely 
5 controlled. 

It is an additional object of the present invention to 
eliminate the installation of remote controlling software 
that is conventionally required for a remote controlling 
terminal and to reduce the load imposed on a system 
*0 manager. 

It is a further object of the present invention to elim- 
inate the management of remote controlling software 
that is conventionally required for a remote controlling 
terminal and to reduce the load imposed on a system 
75 manager. 

It is still another object of the present invention to 
enable a remote control operation that is not dependent 
on the platform provided by a remote controlling termi- 
nal. 

It is a still further object of the present invention to 
reduce the use of resources by a remote controlling ter- 
minal and to reduce the requirements for the hardware 
that can be used for a remote controlling terminal. 

It is yet another object of the present invention to 
provide a fast remote control system that reduces the 
quantity of data that is exchanged by a remote control- 
ling machine and a remote controlled machine during a 
remote control operation. 

It is yet an additional object of the present invention 
to reduce the memory area of a remote controlling ter- 
minal and the load of on a resource, such as a CPU. by 
employing software used for one application as special 
download software for downloading remote controlling 
software. 

Control of a GUI screen at a server in a remote lo- 
cation is obtained while the prior instalment of special 
remote controlling software in a remote controlling ma- 
chine is not required, and remote controlling is per- 
formed by using the obtained software. 

According to the present invention, remote control- 
ling software is linked with and/or described in an HTML 
(HyperText Markup Language) file that is held by a serv- 
er, and a remote controlling machine can easily provide 
remote control without using special software incorpo- 
rated in an arbitrary web browser which supports a JAVA 
applet. 

When the remote controlling software is download- 
ed to the remote controlling machine, it is automatically 
activated and sends a request for the activation of re- 
mote controlled software that is present in a remote con- 
trolled server. The remote controlling machine then ac- 
tivates an event monitor, an event sender, a drawing 
command receiver and a drawing command analyser, 
which are functions of the remote controlling software. 

In response to the request issued by the remote 
controlling machine, the remote controlled server acti- 
vates an event receiver, an event analyser, a drawing 
command monitor and a drawing command sender 
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which are functions of the remote controller software. 
Thus, as is shown in Fig. 2, the same image (the slave 
server display area 220) as that on a display screen 245 
of a slave server 240 is drawn on a web browser screen 
210 at a master controller 100. and the master controller 
100 can remotely control the slave server 240. 

According to a first aspect of the present invention, 
there is provided a remote controlling method of control- 
ling a GUI resource of a server connected to a network 
and having a remote controlling software from a terminal 
connected to the network and having an input device, 
the method comprising the steps of: (a) accessing the 
server through the network according to a server access 
information specified at the terminal; (b) downloading 
the remote controlling software from the server to the 
terminal: (c) sending, from the tenninal. input informa- 
tion that occurs at the input device of the terminal to the 
server according to the downloaded remote controlling 
software; 

(d) receiving, at the server, the sent input information 
from the terminal, and ordering the GUI resource to gen- 
erate a drawing command according to the sent input 
information; (e) sending the drawing command from the 
server to the terminal: and (f ) receiving the drawing com- 
mand at the terminal. 

Reference in this specification to "server access in- 
formation specified at the terminal" involves not only in- 
formation input at a keyboard, but also the concept of 
the selection of a plurality of server access information 
sets that are input in advance. Further, "input informa- 
tion" includes not only signals input by a pointing device, 
such as a mouse : but also keyboard input and voice in- 
put. 

The phrase "ordering the GUI resource to generate 
a drawing command" includes all actions involved in is- 
suing orders having a form that drawing associated soft- 
ware (a window system, a Graphics engine, etc.) 
present in a server understands. A window message 
may be provided for a journal playback hook, as will be 
explained in the preferred embodiment of the present 
invention, or an order may be converted into a serial port 
signal that the server understands, and that is generated 
by a mouse, and the serial port signal may be sent to a 
GUI resource at the server. 

According to a second aspect of the present inven- 
tion, provided is a remote controlling method of control- 
ling a GUI resource of a server connected to a network 
and having a remote controlling software from a terminal 
connected to the network and having an input device 
and a display device, the method comprising the steps 
of: 

(a) accessing the server through the network according 
to a server access information specified at the terminal: 

(b) downloading the remote controlling software from 
the server to the terminal; (c) sending, from the terminal, 
input information that occurs at the input device of the 
terminal to the server according to the downloaded re- 
mote controlling software; (d) receiving, at the server, 
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the sent input information from the terminal, and analys- 
ing the contents of the sent input information; (e) order- 
ing, according to the analysed contents of the input in- 
formation, a drawing engine to generate a drawing com- 
5 mand: (f ) sending the drawing command from the server 
to the terminal; and (g) generating, at the terminal, ac- 
cording to the drawing command received from the serv- 
er, image data to be displayed on a display screen at 
the terminal. 

10 The term "generating image data" is a concept that 
includes not only image data used for a display on a 
screen, but also data before it is sent as image data to 
the display device. 

According to a third aspect of the present invention, 

'5 provided is a remote controlling method of controlling a 
GUI resource of a slave server that exists on the Internet 
and that includes a slave daemon, an HTTP daemon 
and an HTML file and a master applet linked to the 
HTML file, from a master controller having a web brows- 

20 er and including an input device and a display device, 
the remote controlling method comprising the steps of: 
(a) accessing the slave server through the Internet ac- 
cording to a URL designated in the web browser of the 
master controller; (b) downloading the HTML file from 

25 the slave server to the master controller via the HTTP 
daemon: (c) analysing the HTML file at the master con- 
troller and downloading the master applet linked to the 
HTML file: and (d) establishing a connection between 
the slave daemon of the slave server and the master 

30 applet of the master controller. 

According to a fourth aspect of the present inven- 
tion, a server connected to a network comprises: (a) re- 
mote controlled software including an input information 
receiving part, an input information analysing part, a 

35 drawing command monitoring part and a drawing com- 
mand sending part; (b) remote controlling software for 
interacting with the remote controlled software, which 
includes an input information monitoring part, an input 
information sending part, a drawing command receiving 
part and a drawing command analysing part; (c) an 
HTML file holding a description of procedures for ac- 
cessing the remote controlling software; and (d) an HT- 
TP daemon for controlling access to the HTML file from 
the network. 

45 According to a fifth aspect of the present invention, 
a server, connected through a network to a terminal that 
exists on the network, comprises: 
(a) remote controlled software including an input infor- 
mation receiving part, an input information analysing 

50 part, a drawing command monitoring part and a drawing 
command sending part: (b) remote controlling software 
for interacting with the remote controlled software, 
which includes an input information monitoring part, an 
input information sending part, a drawing command re- 

55 ceiving part and a drawing command analysing part; 
and (c) sending means for in response to a request from 
the terminal that the remote controlling software be 
downloaded, sending the remote controlling software to 
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the terminal. 

According to a sixth aspect of the present invention, 
provided is a computer system, which includes a server 
connected to a network and a terminal connected to a 
network, the server comprising: (a) remote controlled s 
software including an input information receiving part, 
an input information analysing part, a drawing command . 
monitoring part and a drawing command sending part; 
(b) remote controlling software for interacting with the 
remote controlled software, which includes an input in- io 
formation monitoring part, an input information sending 
part, a drawing command receiving part and a drawing 
command analysing part: (c) an HTML file holding a de- 
scription of procedures for accessing the remote con- 
trolling software; and (d) an HTTP daemon for control- is 
ling access to the HTML file from the network; and the 
terminal comprising: (e) a web browser for accessing 
the HTML file to download the remote controlling soft- 
ware; and (f) remote controlling software storage for 
storing the remote controlling software. 20 

According to a seventh aspect of the present inven- 
tion, provided is a computer system, which includes a 
server connected to a network and a terminal connected 
to a network, the server comprising: (a) remote control- 
led software including an input information receiving 25 
part, an input information analysing part, a drawing com- 
mand monitoring part and a drawing command sending 
part; (b) remote controlling software for interacting with 
the remote controlled software, which includes an input 
information monitoring part, an input information send- 30 
ing part, a drawing command receiving part and a draw- 
ing command analysing part; and (c) sending means for, 
in response to a request from the terminal that the re- 
mote controlling software be downloaded, sending the 
remote controlling software to the terminal: and the ter- 35 
minal comprising: (d) means for requesting the server 
to download the remote controlling software; and (e) re- 
mote controlling software storage for storing the remote 
controlling software. 

According to an eighth aspect of the present inven- 40 
tion, provided is memory medium comprises: (a) remote 
controlling software, wherein the remote controlling soft- 
ware having an input information monitoring part, an in- 
put information sending part, a drawing command re- 
ceiving part and a drawing command analysing part. 45 
and interacting with remote controlled software having 
an input information receiving part, an input information 
analysing part, a drawing command monitoring part and 
a drawing command sending part; and (b) an HTML file 
holding a description of procedures for accessing the so 
remote controlling software. 

According to a ninth aspect of the present invention, 
provided is a memory medium in which is stored an 
HTML file holding a description of procedures for ac- 
cessing remote controlling software, which includes an ss 
input information monitoring part, an input information 
sending part, a drawing command receiving part and a 
drawing command analysing part, and which interacts 



with remote controlled software that includes an input 
information receiving part, an input information analys- 
ing part, a drawing command monitoring part and a 
drawing command sending part. 

According to a tenth aspect of the present invention, 
provided is a memory medium in which is stored soft- 
ware for sending to a terminal, in response to software 
download request from the terminal, software that in- 
cludes an input information monitoring part, an input in- 
formation sending part, a drawing command receiving 
part and a drawing command analysing part, and that 
interacts with remote controlled software that includes 
an input information receiving part, an input information 
analysing part, a drawing command monitoring part and 
a drawing command sending part. 

How the invention may be carried out will now be 
described by way of example only and with reference to 
the accompanying drawings in which: 

Fig. 1 is a block diagram illustrating an example 
hardware arrangement for a master controller or a slave 
server according to present invention. 

Fig. 2 is a diagram showing an example operating 
state of the present invention. 

Fig. 3 is a block diagram illustrating the arrange- 
ments of a master controller and a slave server accord- 
ing to one embodiment of the present invention. 

Fig. 4 is a block diagram illustrating the arrange- 
ments of the master controller and the slave server ac- 
cording to the embodiment of the present invention. 

Fig. 5 is a diagram for explaining an example HTML 
document used in the embodiment of the present inven- 
tion. 

Fig. 6 is a flowchart showing the operating state of 
a master applet according to the embodiment of the 
present invention. 

Fig. 7 is a flowchart showing the operating state of 
a slave daemon according to the embodiment of the 
present invention. 

Fig. 8 is a conceptual diagram showing the operat- 
ing states of the master applet and the slave daemon 
according to the embodiment of the present invention. 

Fig. 9 is a flowchart showing the processing per- 
formed by an event monitor according to the embodi- 
ment of the present invention. 

Fig. 10 is a flowchart showing the processing per- 
formed by an event sender according to the embodi- 
ment of the present invention. 

Fig. 1 1 is a flowchart showing the processing per- 
formed by an event receiver according to the embodi- 
ment of the present invention. 

Fig. 12 is a flowchart showing the processing per- 
formed by an event analyser according to the embodi- 
ment of the present invention. 

Fig. 13 is a flowchart showing the processing per- 
formed by a drawing command monitor according to the 
embodiment of the present invention. 

Fig. 14 is a flowchart showing the processing per- 
formed by a drawing command sender according to the 
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embodiment of the present invention. 

Fig. 1 5 is a flowchart showing the processing per- 
formed by a drawing command receiver according to the 
embodiment of the present invention. 

Fig. 16 is a flowchart showing the processing per- s 
formed by a drawing command analyser according to 
the embodiment of the present invention. 

Fig. 17 is a diagram showing example data conver- 
sion performed by the event monitor according to the 
embodiment of the present invention. 10 

Fig. 18 is a diagram showing example event data 
according to the embodiment of the present invention. 

Fig. 1 9 is a diagram showing example data conver- 
sion performed by the event analyser according to the 
embodiment of the present invention. is 

Fig. 20 is a diagram showing example drawing com- 
mand data according to the embodiment of the present 
invention. 

Fig. 1 is a schematic diagram illustrating the hard- 
ware arrangement of a remote control machine (master 20 
controller) according to one embodiment of the present 
invention. A master controller 100 includes a central 
processing unit (CPU) 1 and a memory 4. The CPU 1 
and the memory 4 are connected via a bus 2 to a hard 
disk drive 13 that acts as an auxiliary storage device. A 2s 
floppy disk drive (or a memory medium driver for an MO 
or CD-ROM) 20 is connected to the bus 2 via a floppy 
disk controller 1 9. 

A floppy disk (or a memory medium such as an MO 
or a CD-ROM) is inserted into the floppy disk drive 20. 30 
Code for a computer program that interacts with an op- 
erating system and issues commands to the CPU 1 to 
implement the present invention can be recorded on the 
floppy disk or on the hard disk drive 13, or in the ROM 
1 4. The computer program is executed by loading it into 35 
the memory 4. The computer program code can be com- 
pressed, or divided into several segments, and stored 
in a plurality of memory mediums. 

The master controller 100 also includes user inter- 
face hardware, such as a pointing device (a mouse or 40 
a joystick) 7 or a keyboard 6 for input and a display 12 
for providing visual data to a user. In addition, a printer 
can be connected via a parallel port 16 and a modem 
can be connected via a serial port 15. The master con- 
troller 100 can be connected to a network via the serial 45 
port 1 5 and the modem or a communication adapter (an 
Ethernet or Tokenring card) 18 to communicate with an- 
other computer. 

A sound signal obtained by D/A (digital/analogue) 
conversion performed by an audio controller 2 1 is trans- 50 
mitted through an amplifier 22 and is output as sound 
through a loudspeaker 23. The audio controller 21 per- 
forms A/D (analogue/digital) conversion of sound infor- 
mation received via a microphone 24 to fetch sound in- 
formation from outside the system. ss 

It can easily be understood that the master control- 
ler 100 of the present invention may be a common per- 
sonal computer (PC), a workstation, a notebook PC, a 
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palmtop PC, a network computer, an electric home ap- 
pliance incorporating a computer, a game machine hav- 
ing a communication function, or a communication ter- 
minal having a communication function, such as a tele- 
phone, a facsimile machine, a portable telephone or an 
electronic notebook, or it may be constituted by a com- 
bination of these devices. The components in Fig. 1 are 
only examples, and not all of them are required for the 
present invention. 

Particularly, since the hardware arrangement de- 
scribed above is required to operate a remote controlled 
server (slave server), the audio controller 21 , the ampli- 
fier 22, the loudspeaker 23 and the microphone 24, 
which are used for sound processing, are not necessar- 
ily required. The keyboard 6, the mouse 7 and a key- 
board/mouse controller 5, with which direct input by an 
operator is enabled, need only input means for instruct- 
ing the downloading of remote controlling software at 
the slave server to actually perform remote controlling. 

It would be obvious to one having ordinary skill in 
the art that various modification of the arrangement for 
the master controller 1 00 are possible, e.g. . the combin- 
ing of a plurality of machines and the distribution of func- 
tions among them. These modifications are included in 
the concept of the present invention. 

As well as the master controller 100. a slave server 
240 for the present invention can be provided using the 
hardware arrangement shown in Fig. 1. More specifical- 
ly, the slave server 240 need only have a function for 
receiving an event and for converting it into a drawing 
command and sending the command, and a function for 
storing remote controlling software and remote control- 
led software and sending the remote controlling soft- 
ware. Therefore, it can easily be understood that the 
slave server 240 may be a common personal computer 
(PC), a workstation, a notebook PC, a palmtop PC, an 
electric home appliance incorporating a computer, a 
game machine having a communication function, or a 
communication terminal having a communication func- 
tion, such as a telephone, a facsimile machine, a port- 
able telephone or an electronic notebook, or it may be 
constituted by a combination of these devices. 

It should be noted that not all the components in 
Fig. 1 are required for the present invention. Particularly 
since the above described hardware arrangement is re- 
quired for operating a remote controlled server (slave 
server), the audio controller 21, the amplifier 22, the 
loudspeaker 23 and the microphone 24, which are used 
for sound processing, are not necessarily required. In 
addition, the keyboard 6. the mouse 7 and the keyboard/ 
mouse controller 5. with which direct input by an oper- 
ator is possible, are not always required for the remote 
controlled server. 

An operating system for the master controller 100 
can be: one supporting a GUI multi-window environment 
as standard, such as WindowsNT (trademark of Micro- 
soft Corp.), Windows95 (trademark of Microsoft Corp.), 
Windows 3.x (trademark of Microsoft Corp.), OS/2 
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(trademark of IBM Corp.), MacOS (trademark of Apple 
Inc.) or the X-WINDOW system (trademark of MIT) on 
AIX (trademark of IBM): one having a character based 
environment, such as PC-DOS (trademark of IBM 
Corp.) or MS-DOS (trademark of Microsoft Corp.); or an s 
OS installed in a network computer, for example, a real 
time OS such as OS/Open (trademark of IBM CorpO or 
VxWorks (trademark of Wind River Systems, Inc.), or 
JavaOS. In other words, the operating system for the 
master controller 100 is not limited to a specific operat- 10 
ing system environment. 

For one mode of the present invention where re- 
mote controlling software is downloaded by using a web 
browser an arbitrary web browser can be employed so 
long as a Java applet, such as Netscape Navigator is 
(trademark of Netscape Corp. ). Internet Explorer (trade- 
mark of Microsoft Corp.) or Hot Java (trademark of Sun 
Microsystems Corp.), can be used. 

The operating system on the slave server 240 can 
also be: one supporting a GUI multi-window environ- 20 
ment as standard, such as WindowsNT (trademark of 
Microsoft Corp.), Windows95 (trademark of Microsoft 
Corp.), Windows 3.x (trademark of Microsoft Corp.), OS/ 
2 (trademark of IBM Corp.), MacOS (trademark of Apple 
Inc.) or the X-WINDOW system (trademark of MIT) on 25 
AIX (trademark of IBM): or an OS installed in a network 
computer, for example, a real time OS such as OS/Open 
(trademark of IBM Corp.) or VxWorks (trademark of 
Wind River Systems, Inc.), or JavaOS. In other words, 
the operating system for the slave server 240 is not lim- 30 
ited to a specific operating system environment. 

The system configuration of the preferred embodi- 
ment of the present invention will now be described 
while referring to a block diagram in Fig. 3. In the em- 
bodiment, a web browser 21 3 of a master controller 210 35 
works with a Java virtual machine 21 1 . The web browser 
213 accesses the slave server 240 in response to the 
entry of a URL by an operator (the physical input of a 
URL (Uniform Resource Locator) or the selection of a 
URL by pointing at a book mark). 40 

When the slave server 240 is accessed by the mas- 
ter controller 210, an HTTP (Hypertext Transfer Proto- 
col) daemon 241 accesses an HTML file 243 corre- 
sponding to the designated URL. and sends it to the 
master controller 210, The HTTP daemon 241 is a pro- 4s 
gram that provides a service for a client that is accessing 
on the server. As is shown in Fig. 5, the HTML file 243 
contains information 271 linking it to a master applet 
245, which is remote controlling software, so that the 
master applet 245 is sent to the master controller 210. so 

In this invention, since the Industrial standard Inter- 
net browser is employed to download the remote con- 
trolling software, the remote controlling does not depend 
on the platform of a remote control terminal. The HTML 
file 243 may include link information for a plurality of ss 
master applets 245 that exist in different servers or in 
the same server. In this case, the GUIs for a plurality of 
remotely controlled servers are displayed on the web 



browser. However, since the web browser can switch 
applications to send an event in accordance with the po- 
sition of a pointer, a plurality of servers can be remote 
controlled by one browser. In addition, a plurality of 
browsers at the master controller 210 are activated to 
remotely control a plurality of servers. 

When a master applet 215 is downloaded to the 
master controller 210, it is automatically activated and 
sends a connection request to a slave daemon 247 at 
the slave server 240. Upon receipt of the connection re- 
quest, the slave daemon 247 notifies the master applet 
215 at the master controller 210 that the connection is 
enabled. Fig. 4 is a diagram showing the condition 
where the slave daemon 247 of the slave server 240 
has established a connection. 

Although an explanation for all the functional blocks 
in Figs. 3 and 4 has been given, these functional blocks 
are logical functional blocks, and as such are not spe- 
cifically provided by a hardware set or a software set, 
but can be provided by a compound hardware and soft- 
ware set. or by a hardware and a software set function- 
ing in common. 

Furthermore, not all of the functional blocks in Figs. 
3 and 4 are always required for the present invention. 
When the present invention employs a method for 
downloading software that does not include the use of 
a web browser, the Java virtual machine 211 and the 
web browser 21 3 can be replaced by other software that 
requests the remote controlling software. And at the 
slave server 240, the HTML file 243 is also not abso- 
lutely required; the HTTP daemon 241 can be replaced 
by software that, in response to a request from the mas- 
ter controller 210, sends the remote controlling software 
to the master controller 210. 

The processing for establishing a connection be- 
tween remote controlling software and remote control- 
led software and for preparing to perform the remote 
controlling function will now be explained while referring 
to the flowcharts in Figs. 6 and 7, and the conceptual 
diagram showing the operating states of the master ap- 
plet 215 and the slave daemon 247 in Fig. 8. 

First, in Fig. 6, when the master applet 21 5 is down- 
loaded to the master controller 210, the master applet 
215 is automatically activated (block 401). After being 
activated, the master applet 215 issues a connection re- 
quest to the slave daemon 247. 

In Fig. 7, at this time, the slave daemon 247 at the 
slave server 240, rendered active in advance (block 
501), is monitoring signal sending for a connection re- 
quest issued by the master applet 215 (block 503). 
When the connection request from the master applet 
21 5 is detected, the slave daemon 247 notifies the mas- 
ter applet 215 that the connection is enabled, activates 
a drawing command monitor 323 and a drawing com- 
mand sender 329 (blocks 505 and 507). and redraws 
the GUI screen (block 509). Fig. 8 is a conceptual dia- 
gram showing the operating stales of the master applet 
and the slave daemon according to the embodiment of 
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the present invention. 

The GUI screen is entirely redrawn because the 
current GUI screen of the slave server 240 is sent to the 
master controller 210. Following this action, an event re- 
ceiver 311 and an event analyser 315 are activated. In s 
the embodiment of the present invention, threads are 
allocated individually to the drawing command monitor 
323, the drawing command sender 329, the event re- 
ceiver 31 1 and the event analyser 31 5 to enable parallel 
operation. w 

Upon receipt of the connection enable notice from 
the slave server 240, the master applet 215 assumes 
that the connection is successful (block 405 in Fig. 6), 
and activates an event sender 307. a drawing command 
receiver 333, an event monitor 303 and a drawing com- is 
mand analyser 337 (blocks 407, 409, 411 and 413). In 
this embodiment, as well as being allocated for the stave 
daemon 247, threads are allocated individually for the 
event sender 307, the drawing command receiver 333, 
the event monitor 303 and the drawing command ana- 20 
lyser 337 to enable parallel operation. 

The remote controlling processing in this embodi- 
ment will now be described while referring to flowcharts 
. in Figs. 9 through 16 and to the operating states of the 
master applet 215 and the slave daemon 247 in Fig. 8. 2s 
Fig. 9 is a flowchart showing the processing for the event 
monitor 303. The event monitor 303 monitors events at 
the master applet 215 of the master controller 210 that 
occur when an input device, such as a mouse or a key- 
board, is operated (block 421). 30 

In the preferred embodiment of the present inven- 
tion, as is shown in Fig. 17, a signal for operating an 
input device, such as a mouse or a keyboard, is input 
as an event type, a coordinate value for the event, and 
a key number to the event monitor 303 by using a nan- 35 
dleEvent function 301. In this embodiment, when the 
slave daemon 247 is activated, the GUI screen of the 
slave server 240 is re-drawn (block 509 in Fig. 7). and 
the current GUI screen is sent as a drawing command 
to the master controller 210. 40 

Upon receipt of the drawing command, the master 
controller 21 0 allocates a predetermined area to the GUI 
screen that is based on the sent drawing command. The 
event monitor 303 monitors user input to the allocated 
area (a slave server display area 220 in Fig. 2). The op- 4s 
erator can perform remote controlling not only by using 
a pointing device, such as a mouse, to input data to the 
slave server display area 220, but also by using key- 
board input and sound input while the slave server dis- 
play area 220 is focused. so 

In this embodiment, to reduce the quantity of the 
data that is sent to the slave daemon 247, the event 
monitor 303 converts the data into signals that provides 
the minimum information necessary for the slave dae- 
mon 247 to generate a window message, which will be ss 
described later. It is also possible that the converted da- 
ta is further compressed or encoded and that the result- 
ant data is sent to the slave daemon 247, which decom- 
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presses the data. 

The data conversion performed by the event mon- 
itor 303 is not an indispensable process, and event data 
that are generated by handle Event function 301 may 
be sent unchanged to the slave daemon 247. Fig. 18 is 
a diagram showing common event data examples that 
are obtained by conversion in this embodiment. 

In the embodiment, since the event monitor 303 and 
the event sender 307 can be independently operated, 
an event provided by conversion is stored temporarily 
in an event buffer 305 (block 423). As the event buffer 
305 will be also become unnecessary if each time an 
event occurs it is sent directly to the slave daemon 247, 
the event buffer 305 is not a requisite component for the 
present invention. The processing is continued until the 
operation is terminated (block 425). When it is ascer- 
tained that the operation has been terminated, the event 
monitor 303 is deactivated (block 427). 

On the other hand, as is shown in Fig. 10, when the 
event sender 307 is activated, an operation start event 
is sent to the slave daemon 247 (block 451). Then, a 
check is per formed to determine whether or not an 
event is stored in the event buffer 305 (block 453). When 
an event is stored in the event buffer 305, it is sent to 
the slave daemon 247 (block 455). In addition to the 
event monitor 303, the event sender 307 continues the 
performance of the processing until the operation is ter- 
minated (block 457). When the termination of the oper- 
ation is detected, a signal to that effect is sent to the 
respective sections (block 459). and the event sender 
307 is deactivated (block 461). 

Fig. 11 is a flowchart showing the processing per- 
formed by the event receiver 311. When the event re- 
ceiver 311 receives an operation start event from the 
master applet 215, it ascertains whether acceptance of 
the event is possible (block 521). When the event re- 
ceiver 31 1 accepts the event from the master applet 215 
(block 523), it stores the event in an event buffer 31 3 for 
the slave daemon 247 (block 525). At the event receiver 
311, this process is repeated until the operation is ter- 
minated (block 527). When the termination of the oper- 
ation is detected, the event receiver 311 is deactivated 
(block 529). 

As is shown in Fig. 1 2, the event analyser 31 5 mon- 
itors the storing of an event in the event buffer 31 3 (block 
531 ), and analyses the event stored in the event buffer 
313 (block 533). 

In this embodiment, as is shown in Fig. 1 9, the event 
analyser 315 analyses the received event, converts it 
into a window message having a form that is compatible 
with a window system 320, which works on the slave 
server 240, and transmits the message via a journal 
playback hook 317 to the window system 320 (block 
541). 

The journal playback hook 317 is generally paired 
with a journal record hook when it is used. The journal 
record hook can record a window message input by the 
operator, and the journal playback hook 317 can repro- 



EP 0 878 759 A1 



13 

duce the recorded window message. 

In this embodiment, the function of the journal play- 
back hook 31 7 is employed. The window message gen- 
erated by the event analyser 31 5 is sent to the journal 
playback hook procedure 317, and the window system s 

320 can handle it as a window message occurring in the 
window system 320. The window system 320 interprets 
the received window message, determines what is to be 
input to which application, and causes the GUI screen 

to be rewritten. The GUI screen that is rewritten in re- 10 
sponse to the window message is output as an image 
on a physical display device 245 to a Graphics engine 

321 and a display driver 325 in the window system 320. 
In the event analyser 315, the above process is re- 
peated until the operation is terminated (block 535). is 

When the termination of the operation is detected, 
the event analyser 315 forwards a signal to that effect 
to the respective sections (block 537). The event ana- 
lyser 315 is thereafter deactivated (block 539). 

Fig. 13 is a flowchart showing the processing per- 20 
formed by the drawing command monitor 323. The 
drawing command monitor 323 is situated between the 
Graphics engine 321 and the display driver 325 of the 
window system 320, and can act as the display driver 
325 with respect to the Graphics engine 321 , while act- 2s 
ing as the Graphics engine 321 with respect to the dis- 
play driver 325. 

The drawing command monitor 323, which is locat- 
ed between the Graphics engine 321 and the display 
driver 325, monitors all the APIs that are called for re- 30 
writing the GUI screen at the slave server 240, and pre- 
pares a drawing command based on the API (block 
563). The drawing command is stored in a drawing com- 
mand buffer 327 in the slave server 240 (block 565). Fig. 
20 is a diagram showing example data for a typical draw- 35 
ing command in this embodiment. 

It is possible for a converted drawing command to 
be further compressed or encoded, and for the resultant 
command to be sent to the master applet 215, which 
thereafter decom presses it. 40 

The conversion performed by the drawing com- 
mand monitor 323 is not essential, and data that are 
generated by the Graphics engine 321 and sent to the 
display driver 325 may be sent unchanged to the master 
applet 215. 45 

Fig. 14 is a flowchart showing the processing per- 
formed by the drawing command sender 329. The draw- 
ing command sender 329 monitors the storing of draw- 
ing commands in the drawing command buffer 327 
(block 551 ). When the drawing command sender 329 so 
detects the storage of a drawing command in the draw- 
ing command buffer 327. it sends the drawing command 
to the master controller 210 (block 553). This process is 
repeated until the operation is terminated (block 555). 
When the termination of the operation is detected, the $5 
drawing command sender 329 is deactivated (block 
557). 

Fig. 15 is a flowchart showing the processing per- 
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formed by the drawing command receiver 333. The 
drawing command receiver 333 monitors the sending of 
drawing commands from the slave daemon 247 (blocks 
441 and 443), and stores received drawing commands 
in a drawing command buffer 335 in the master control- 
ler 210 (block 445). This processing is repeated by the 
drawing command receiver 333 until the operation is ter- 
minated (block 447). When the termination of the oper- 
ation is detected, the drawing command receiver 333 is 
deactivated (block 449). 

Fig. 16 is a flowchart showing the processing per- 
formed by the drawing command analyser 337. The 
drawing command analyser 337 monitors the storing of 
drawing commands in the drawing command buffer 335 
(block 431 ). The drawing command analyser 337 anal- 
yses the stored drawing commands to generate the 
same image as that drawn on the GUI screen of the 
slave server 240, and draws that image on the web 
browser screen 220 of the master controller 210 (block 
433). The above process is repeated by the drawing 
command analyser 337 until the operation is terminated 
(block 435). when the termination of the operation is de- 
tected, the drawing command analyser 337 is deactivat- 
ed (block 437). 

As is described above, according to the present in- 
vention, provided is a computer system wherein, even 
when no remote controlling software is currently availa- 
ble at a remote control terminal, remote controlling of a 
server can be effected simply by accessing it. 

Further, according to the present invention, it is pos- 
sible to simplify the upgrading operation, for remote con- 
trolling software held by a remote control terminal that 
accompanies the upgrading of the version of the remote 
controlling software held by a server that is to be remote 
controlled. 

In addition, according to the present invention, it is 
possible to eliminate the installation of remote control- 
ling software that is conventionally required for a remote 
control terminal and to reduce the load imposed on a 
system manager. 

Furthermore, according to the present invention, it 
is possible to eliminate the management of remote con- 
trolling software that is conventionally required for a re- 
mote control terminal and to reduce the load imposed 
on a system manager. 

Moreover, according to the present invention, it is 
possible to enable a remote controlling operation that is 
not dependent on the platform provided for a remote 
control terminal. 

Further, according to the present invention, it is pos- 
sible to reduce the use of resources by a remote control 
terminal and to reduce the requirements for the hard- 
ware that can be used for a remote control terminal. 

In addition, according to the present invention, pro- 
vided is a fast remote controlling system that reduces 
the quantity of data that is exchanged by a remote con- 
trolling machine and a remotely controlled machine dur- 
ing a remote controlling operation. 
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Furthermore, according to the present invention, it 
is possible to reduce the memory area of a remote con- 
trol terminal and the load of on a resource, such as a 
CPU, by employing software used for one application 
as special download software for downloading remote s 
controlling software. 

Claims 

10 

1 . A remote controlling method of controlling a GUI re- 
source of a server (240) connected to a network 
(230) and having a remote controlling software from 
a terminal connected to the network and having an 
input device, said method comprising the steps of: is 

(a) accessing said server through said network 
according to a server access information spec- 
ified at said terminal: 

(b) downloading said remote controlling soft- 20 
ware from said server to said terminal: 

(c) sending, from said terminal, input informa- 
tion that occurs at said input device of said ter- 
minal to said server according to the download- 
ed remote controlling software; 25 

(d) receiving, at said server, the sent input in- 
formation from said terminal, and ordering said 
GUI resource to generate a drawing command 
(255) according to the sent input information; 

(e) sending said drawing command from said 30 
server to said terminal: and 

(f) receiving said drawing command at said ter- 
minal. 

2. A remote controlling method of controlling a GUI re- 35 
source of a server connected to a network and hav- 
ing a remote controlling software from a terminal 
connected to the network and having an input de- 
vice and a display device (11), said method com- 
prising the steps of: 40 

(a) accessing said server through said network 
according to a server access information spec- 
ified at said terminal; 

(b) downloading said remote controlling soft- «5 
ware from said server to said terminal: 

(c) sending, from said terminal, input informa- 
tion that occurs at said input device of said ter- 
minal to said server according to the download- 
ed remote controlling software: 50 

(d) receiving, at said server, the sent input in- 
formation from said terminal, and analysing the 
contents of the sent input information: 

(e) * ordering, according to said analysed con- 
tents of said input information, a drawing en- ss 
gine to generate a drawing command; 

(f) sending said drawing command from said 
server to said terminal; and 



(g) generating, at said terminal according to 
said drawing command received from said 
server, image data to be displayed on a display 
screen at said terminal. 

3. A remote controlling method of controlling a GUI re- 
source of a slave server (240) that exists on the In- 
ternet (230) and that includes a slave daemon 
(247), an HTTP daemon (241) and an HTML file 
(243) and a master applet linked to the HTML file, 
from a master controller (210) having a web brows- 
er (21 3) and including an input device and a display 
device (11), said remote controlling method com- 
prising the steps of: 

(a) accessing said slave server (240) through 
the Internet (230) according to a URL designat- 
ed in said web browser (213) of said master 
controller; 

(b) downloading said HTML file from said slave 
server (240) to said master controller via said 
HTTP daemon: 

(c) analysing said HTML file at said master con- 
troller and downloading said master applet 
(21 5) linked to said HTML file: and 

(f) establishing a connection between said 
slave daemon (247) of said slave server (240) 
and said master applet (245) of said master 
controller. 

4. A server (240) connected to a network (230) com- 
prising: 

(a) remote controlled software including an in- 
put information receiving part, an input informa- 
tion analysing part, a drawing command moni- 
toring part and a drawing command sending 
part; 

(b) remote controlling software for interacting 
with said remote controlled software, which in- 
cludes an input information monitoring part, an 
input information sending part, a drawing com- 
mand receiving part and a drawing command 
analysing part: 

(c) an HTML file (243) holding a description of 
procedures for accessing said remote control- 
ling software; and 

(d) an HTTP daemon (241) for controlling ac- 
cess to said HTML file from said network. 

5. A server (240). connected through a network (230) 
to a terminal that exists on said network, compris- 
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ing: 

(a) remote controlled software including an in- 
put information receiving part, an input informa- 
tion analysing part, a drawing command moni- s 
toring part and a drawing command sending 
part: 

(b) remote controlling software for interacting 
with said remote controlled software, which in- 
cludes an input information monitoring part, an 10 
input information sending part, a drawing com- 
mand receiving part and a drawing command 
analysing part: and 

(c) sending means for. in response to a request 
from said terminal that said remote controlling is 
software be downloaded, sending said remote 
controlling software to said terminal. 

A computer system, which includes a server (240) 
connected to a network (230) and a terminal con- 20 
nected to a network, said server comprising: 

(a) remote controlled software including an in- 
put information receiving part, an input informa- 
tion analysing part, a drawing command moni- 25 
toring part and a drawing command sending 
part; 

(b) remote controlling software for interacting 
with said remote controlled software, which in- 
cludes an input information monitoring part, an 30 
input information sending part, a drawing com- 
mand receiving part and a drawing command 
analysing part; 

(c) an HTML file (243) holding a description of 
procedures for accessing said remote control- 35 
ling software: and 

(d) an HTTP daemon (241) for controlling ac- 
cess to said HTML file from said network; and 
said terminal comprising: 

(e) a web browser (213) for accessing said 40 
HTML file to download said remote controlling 
software: and 

(f) remote controlling software storage for stor- 
ing said remote controlling software. 

45 

A computer system, which includes a server (240) 
connected to a network (230) and a terminal con- 
nected to the network, said server comprising: 

(a) remote controlled software including an in- so 
put information receiving part, an input informa- 
tion analysing part, a drawing command moni- 
toring part and a drawing command sending 
part: 

(b) remote controlling software for interacting ss 
with said remote controlled software, which in- 
cludes an input information monitoring part, an 
input information sending part, a drawing com- 



mand receiving part and a drawing command 
analysing part; and 

(c) sending means for, in response to a request 
from said terminal that said remote controlling 
software be downloaded, sending said remote 
controlling software to said terminal: 

and said terminal comprising: 

(d) means for requesting said server to down- 
load said remote controlling software: and 

(e) remote controlling software storage for stor- 
ing said remote controlling software. 

8. A memory medium comprising: 

(a) remote controlling software, wherein said 
remote controlling software having an input in- 
formation monitoring part, an input information 
sending part, a drawing command receiving 
part and a drawing command analysing part, 
and interacting with remote controlled software 
having an input information receiving part, an 
input information analysing part, a drawing 
command monitoring part and a drawing com- 
mand sending part; and 

(b) an HTML file (243) holding a description of 
procedures for accessing said remote control- 
ling software. 

9. A memory medium in which is stored an HTML file 
(243) holding a description of procedures for ac- 
cessing remote controlling software, which includes 
an input information monitoring part, an input infor- 
mation sending part, a drawing command receiving 
part and a drawing command analysing part, and 
which interacts with remote controlled software that 
includes an input information receiving part, an in- 
put informatbn analysing part, a drawing command 
monitoring part and a drawing command sending 
part. 

10. A memory medium in which is stored software for 
sending to a terminal, in response to software 
download request from said terminal, software that 
includes an input information monitoring part, an in- 
put information sending part, a drawing command 
receiving part and a drawing command analysing 
part, and that interacts with remote controlled soft- 
ware that includes an input information receiving 
part, an input information analysing part, a drawing 
command monitoring part and a drawing command 
sending part. 
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